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Executive  Summary 

TARGETING  BEST  VALUE  IN  PERSONAL 
PROPERTY  PROCUREMENT 


The  Department  of  Defense  is  changing  its  procurement  practices.  Tradi¬ 
tionally  focused  on  low  price,  it  is  now  striving  to  obtain  the  best  value. 

The  Military  Traffic  Management  Command  (MTMC)  is  one  of  many  Defense 
activities  developing  a  best-value  procurement  program.  As  manager  of  the  DoD’s 
Personal  Property  Shipment  and  Storage  Program,  which  in  FY91  entailed 
188,000  domestic  shipments  involving  some  1,100  commercial  carriers  at  a  cost  of 
more  than  $284  million,  MTMC  seeks  to  award  personal  property  shipments  to 
commercial  carriers  by  taking  into  account  both  the  linehau’  transportation  rr  i.es 
they  bid  and  the  quality  of  the  services  they  provide. 

Building  upon  successful  best-value  procurement  programs  in  both  the  public 
and  private  sectors,  we  recommend  that  MTMC  use  indexes  to  determine  the  overall 
value  provided  by  personal  property  carriers.  Two  indexing  methods  are  particularly 
promising.  I  he  first,  Best  Value  Rate,  keys  on  four  performance  measures:  on-time 
pickup,  on-time  delivery,  loss  and  damage,  and  carrier  turnbacks  or  DoD  pullbacks. 
Using  the  statistical  technique  of  principal  component  analysis,  MTMC  would  assign 
weights  to  each  of  these  performance  measures  and  then  mathematically  apply  them 
against  a  carrier’s  bid  r^te  to  derive  a  Best  Value  Rate.  The  second  indexing  method. 
Best  Value  Score,  is  similar  to  the  first  but  differs  in  that  MTMC  would  assign  a 
weight  to  the  carrier’s  bid  as  well  as  to  each  of  the  four  performance  factors. 

Although  the  use  of  either  indexing  method  will  satisfy  M'TMC’s  requirements 
for  a  best-value  procurement  program,  we  believe  the  Best  Value  Score  method  has 
two  primary  advantages.  It  is  more  direct  because  it  requires  only  one  computation, 
and  it  does  not  assume  a  direct  relationship  between  quality  and  bid  rates  which, 
under  some  circumstances,  may  be  inequitable  to  personal  property  carriers. 

Regardless  of  which  indexing  method  it  chooses,  MTMC  also  will  need  to  change 
its  practices  for  allocating  shipments  to  carriers.  The  current  procurement  process 
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tends  to  array  carriers  into  rate  groups,  with  MTMC  then  awarding  traffic  equitably 
to  all  carriers  in  a  particular  rate  group.  The  very  nature  of  best- value  procurement, 
however,  will  eliminate  rate  groupings.  Instead,  MTMC  will  generate  a  continuum 
of  Best  Value  Rates  or  Scores  for  its  carriers.  Because  it  is  not  practical  to  award  all 
traffic  to  the  best-value  carrier,  we  propose  that  MTMC  use  Best  Value  Clusters. 
Under  this  concept,  MTMC  would  award  the  most  shipments  to  the  top  cluster  of 
best-value  carriers.  The  lower  clusters  of  best-value  carriers  would  be  awarded 
smaller  shares  of  shipments.  This  approach  has  the  principal  advantage  of  providing 
incentives  for  all  carriers  to  continue  to  improve  their  performance. 

To  demonstrate  the  potential  of  these  best-value  alternatives,  we  recommend 
that  MTMC  conduct  a  test  at  selected  personal  property  shipping  offices.  The  results 
of  this  test  will  enable  MTMC  to 

•  Select  a  best-value  indexing  method 

•  Adopt  enhanced  shipment  allocation  practices 

•  Develop  an  automated  systems  capability  to  support  best-value  procurement 
in  personal  property 

•  Identify  cost  savings  and  quality  improvements 

•  Decide  whether  best  value  meets  its  requirements. 

The  best-value  alternative  that  we  recommend  is  a  dramatic  departure  from 
how  MTMC  traditionally  procures  personal  property  transportation  services.  While 
appearing  to  hold  promise  of  being  the  standard-bearer  for  the  procurement  of  all 
transportation  services  in  the  future,  MTMC  needs  to  first  demonstrate  its 
practicality,  utility,  and  cost-effectiveness. 
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CHAPTER  1 

QUAUTY  MAKES  A  DIFFERENCE 


INTRODUCTION 

In  1988,  the  following  statement  appeared  in  an  American  Society  for  Quality 
Control  publication: 

. . .  quoted  price  alone  is  no  longer  a  viable  means  of  source  selection  because 
the  eleme'its  of  quality,  delivery,  and  service  overshadow  the  initial  low  bid. 

A  prod’-.ct  that  is  not  delivered  on  schedule,  or  that  is  defective,  is  not  a 
bargr  .n  at  any  price — 1 

Although  many  organizations  subscribe  to  this  position,  they  have  had 
difficulty  incorporating  quality  considerations  into  their  procurement  practices.  The 
Department  of  Defense  is  one  of  those  organizations.  In  the  movement  and  storage  of 
personal  property,  for  example,  DoD  activities  award  shipments  to  commercial 
carriers  on  the  basis  of  lowest  overall  cost  (i.e.,  a  combination  of  the  rate  bid  and 
quality  of  service),  but  their  primary  emphasis  is  on  the  carriers’  rates. 

The  Military  Traffic  Management  Command  (MTMC),  which  manages  DoD’s 
Personal  Property  Shipment  and  Storage  Program,  seeks  to  change  that  emphasis. 
According  to  a  16  July  1992  policy  announcement  by  the  Commander,  MTMC,  it 
wants  to  obtain  the  “best  value”  for  DoD's  personal  property  expenditures.2 

. . .  Best  value  acquisition  is  preferred  and  will  be  used  when  appropriate. 

Where  regulatory  roadblocks  to  using  best  value  exist,  action  will  be  taken 
to  pursue  their  elimination. . . . 

This  report  describes  a  methodology  and  identifies  alternative  approaches  that 
MTMC  could  use  to  award  personal  property  shipments  to  commercial  carriers  based 
upon  best  value.  Before  presenting  that  methodology,  we  provide  an  overview  of 
DoD’s  personal  property  program.  We  also  examine  whether  added  emphasis  on 


lAmerican  Society  for  Quality  Control,  Vendor-Vendee  Technical  Committee,  Procurement 
Quality  Control,  4th  edition,  ASQC  Quality  Press,  1986. 

2From  the  Commander,  Military  Traffic  Management  Command  (M'fMC),  Subject:  MTMC 
Policy  for  Acquisition  of  Transportation  Services,  16  July  1992. 
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quality  of  service  in  the  procurement  process  could  affect  personal  property  shipment 
awards  to  individual  carriers. 

OVERVIEW  OF  THE  PERSONAL  PROPERTY  PROGRAM 

Foundation 

The  MTMC  manages  the  DoD’s  Personal  Property  Shipment  and  Storage 
Program  on  behalf  of  the  Assistant  Secretary  of  Defense  (Production  and  Logistics), 
under  authority  of  DoD  Directive  4500.34,  DoD  Personal  Property  Shipment  and 
Storage  Program,  10  April  1986.  That  Directive  also  authorizes  publication  of 
DoD  4500. 34-R,  Personal  Property  Traffic  Management  Regulation  (PPTMR), 
October  1991,  which  prescribes  the  day-to-day  operating  procedures. 

Every  6  months,  MTMC  receives  more  than  1.1  million  rate  bids  for  domestic 
movements  from  commercial  carriers.  Carriers  submit  their  rates  for  specific 
domestic  traffic  lanes  (routes)  in  response  to  semiannual  rate  solicitations  from 
MTMC.  They  express  their  rates  as  a  percent  of  MTMC’s  baseline  linehaul 
transportation  rate  schedule,  which  is  published  in  Domestic  Personal  Property  Rate 
Solicitation  D-l. 

Personal  Property  Operations 

Although  the  DoD’s  domestic  Personal  Property  Shipment  and  Storage 
Program  functions  under  centralized  MTMC  management,  its  operations  are  carried 
out  at  153  Military  Service-managed  Personal  Property  Shipping  Offices  (PPSOs).3 
The  PPSOs  use  the  services  of  approximately  1,100  commercial  carriers  each  year  to 
move  the  household  goods,  xinaccompanied  baggage,  mobile  homes,  privately  owned 
vehicles,  and  personal  firearms  of  military  members  throughout  the  United  States. 
In  FY91,  DoD  made  over  188,000 domestic,  door-to-door,  motor  van  (Code  1) 
shipments  comprising  almost  one  billion  pounds  of  personal  property,  at  a  total  cost 
exceeding  $284  million.  The  program’s  size  and  scope  are  unmatched  in  either 


3The  153  PPSOs  in  the  United  States  consist  of  45  Army,  30  Navy,  69  Air  Force,  and  9  Marine 
Corps.  The  Coast  Guard  also  operates  21  PPSOs. 
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private  industry  or  other  Federal  agencies.  Table  1-1  presents  a  summary  of  the 
program.4 


TABLE  1-1 

DoD  PERSONAL  PROPERTY  MOVEMENT  DATA 
(FY91) 


Military  Service 

Shipments 

(thousands) 

Weight 
(millions 
of  pounds) 

Cost 
(millions 
of  dollars) 

Army 

62 

295 

88 

Navy 

59 

291 

84 

Air  Force 

53 

274 

90 

Marine  Corps 

13 

65 

21 

Other  OoO 

1 

4 

1 

Total 

188 

929 

284 

Every  domestic  PPSO  maintains  Traffic  Distribution  Records  (TDR?'  on  which 
it  records  shipment  awards.5  According  to  the  PPTMR,  at  the  end  of  each  year,  all 
carriers  in  a  particular  rate  group  should  have  received  shipments  that  total  within 
20,000  poimds  of  each  other. 

The  TDR  lists  carriers  according  to  their  filed  rates,  with  the  lowest  rate 
carriers  listed  first.  In  an  attempt  to  incorporate  some  consideration  for  quality  into 
the  award  of  personal  property  shipments,  MTMC  established  the  Total  Quality 
Assessment  Program  (TQAP). 


^Although  we  limit  our  discussion  in  this  report  to  domestic  Code  1  personal  property 
shipments,  we  believe  that  our  conclusions  and  recommendations  could  also  apply  to  DoD’s  worldwide 
program.  In  FY91,  that  program  totaled  0.6  million  shipments;  more  than  1.7  billion  pounds;  and 
transportation,  storage,  loss/damage  payments  to  members,  and  other  related  fees  of  approximately 
S2.2  billion. 

5The  PPSOs  maintain  separate  TDRs  for  each  destination  state  and  the  District  of  Columbia  in 
the  continental  United  States.  They  also  establish  separate  TDRs  for  each  area  of  operation  within  a 
destination  PPSO  s  area  of  responsibility.  Carriers  offer  rates  on  the  routes  they  wish  to  serve,  with  a 
route  being  generally  from  an  origin  PPSO’s  area  of  responsibility  to  one  of  the  48  contiguous  states  or 
Alaska.  The  PPTMR  provides  a  more  complete  description  of  the  DoD’s  traffic  distribution  rules  and 
requirement.s. 
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Under  the  provisions  of  that  program,  carriers  are  sequenced  within  each  rate 
grouping  on  a  TDR  according  to  their  average  TQAP  score.6  That  process  results  in 
the  low-rate  carrier  with  the  highest  score  being  awarded  traffic  first,  with  other 
carriers  offering  the  same  low  rate  but  having  lower  TQAP  scores  awarded  traffic  in 
turn.  Unless  a  route  has  a  particularly  heavy  traffic  flow,  only  those  carriers  offering 
the  lowest  rates  are  typically  awarded  shipments,  particularly  during  the  low-traffic 
season  (October  through  April). 

Responsibilities 

The  personal  property  program  requires  extensive  coordination  and  cooperation 
among  MTMC,  PPSOs,  property  owners,  and  commercial  carriers.  Each  has 
important  responsibilities  to  ensure  timely  and  effective  transport  of  a  member’s 
personal  property.  Those  responsibilities  are  summarized  below. 

MTMC 

The  MTMC  provides  overall  technical  and  management  direction  for  the 
program  in  accordance  with  DoD  policy  guidance.  Its  specific  responsibilities  include 
approving  carriers  for,  and  removing  carriers  from,  program  participation;  soliciting, 
negotiating,  assessing,  and  accepting  carriers’  rates;  and  establishing  performance 
standards  and  measuring  carrier  performance  against  those  standards. 

PPSOs 

The  PPSOs  perform  a  variety  of  functions  in  dealing  with  both  military 
members  and  carriers  on  a  day-to-day  basis.  They  approve  carriers  for  service,  obtain 
all  required  transportation  and  storage  services  for  military  members  located  within 
designated  areas,  distribute  shipments  among  eligible  carriers,  monitor  and  evaluate 
carrier  performance,  and  take  disciplinary  actions  against  carriers  that  provide 
deficient  service. 


6The  MTMC  has  established  performance  standards  for  on-time  pickup  (20  points),  on-time 
delivery  (40  points),  and  absence  of  loss  or  damage  (40  points).  The  TQAP  score  is  simply  the  average 
of  the  points  earned  by  a  carrier  over  all  shipments  serviced.  A  carrier  must  attain  an  average  score  of 
at  least  90  during  a  6-month  evaluation  period  to  remain  eligible  for  shipment  award  in  the  next  cycle. 
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Property  Owners 

The  owners  of  the  personal  property  are  responsible  for  notifying  their  PPSOs 
upon  receipt  of  movement  orders  and  complying  with  all  guidance  provided  during 
the  counseling  sessions. 

Commercial  Carriers 

The  commercial  carriers  play  a  pivotal  role  in  ensuring  the  safe  and  timely 
pickup,  packing,  transport,  delivery,  and  unpacking  of  the  members’  personal 
property.  The  quality  of  their  service  is  the  major  contributor  to  each  member’s 
satisfaction  with  his  or  her  move. 

Supporting  Systems 

The  MTMC  and  PPSOs  use  three  automated  systems  in  carrying  out  their 
personal  property  responsibilities; 

•  Transportation  Operational  Personal  Property  Standard  System  (TOPS), 
which- automates  the  PPSO’s  operations 

•  Worldwide  Household  Goods  Information  System  for  Transportation 
(WHIST),  which  is  MTMC’s  rate  and  shipment  information  database  for 
personal  property  movements 

•  TQAP,  which  MTMC  uses  to  evaluate  carrier  performance. 

Building  upon  this  overview  of  the  DoD’s  Personal  Property  Shipment  and 
Storage  Program,  we  now  examine  whether  an  expanded  consideration  of  quality  in 
awarding  personal  property  shipments  could  affect  the  allocation  of  shipments 
among  carriers. 

POTENTIAL  EFFECTS  OF  QUALITY 

To  determine  the  potential  effects  of  quality  on  personal  property  trans¬ 
portation  service  procurements,  we  analyzed  the  rate  and  quality  records  of 
22  randomly  selected  carriers  serving  the  Joint  Personal  Property  Shipping  Office  in 
Massachusetts  (JPPSOMA)  to  five  destination  states  (California,  Georgia,  Illinois, 
New  York,  and  Texas).  (We  used  the  rates  in  effect  during  the  current  cycle  and 
quality  data  from  the  summer  1992  evaluation  cycle.)  In  our  analysis,  we  sought  to 
determine  the  extent  to  which  carrier  rankings  could  change  when  their  rates  were 
adjusted  for  quality.  Although  the  methodology  we  used  is  general  and  not  what  we 
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advocate  later  in  this  report,  it  did  serve  to  establish  %vhether  considering  quality 
might  impact  procurement. 

Methodology 

Carriers  submit  their  rates,  expressed  as  a  percent  of  MTMC’s  “Baseline 
Linehaul  Transportation  Rate  Schedule,”  for  the  specific  routes  they  wish  to  service. 
The  TQAP  scores,  however,  reflect  overall  carrier  performance  from  an  origin  PPSO 
(the  scores  are  not  route  specific).  To  attain  some  compatibility  between  the  rates 
and  the  quality  scores,  we  averaged  each  carrier’s  rates  for  the  five  routes.  For 
example,  if  a  carrier  bid  78  percent  to  California,  75  percent  to  Georgia,  80  percent  to 
Illinois,  100  percent  to  New  York,  and  78  percent  to  Texas  from  JPPSOMA,  its 
average  bid  rate  was  82.2  percent,  or  .822. 

Then  we  calculated  the  following; 

•  The  average  quality  score  (93,337)  for  all  carriers  serving  JPPSOMA 

•  A  normalized  quality  score  for  each  carrier,  by  dividing  the  average  quality 
score  into  the  carrier’s  TQAP  score 

•  An  adjusted  quality  rate  for  each  carrier,  by  dividing  its  normalized  quality 
score  into  its  average  bid  rate. 

The  higher  a  carrier’s  normalized  quality  score,  the  more  value  in  terms  of 
quality  it  provides  the  DoD.  Higher  normalized  quality  scores  also  lower  the  adjusted 
quality  rate,  which  we  used  to  rank  the  carriers  in  terms  of  both  rates  and  quality. 
(Now,  carriers  are  ranked  and  awarded  shipments  according  to  the  sequence  in  which 
they  appear  on  TDRs.) 

Results 

Table  1-2  presents  the  details  for  each  of  the  22  carriers,  while  Table  1-3  shows 
the  carrier  rankings,  both  by  average  bid  rate  alone  and  then  by  average  bid  rate 
adjusted  for  quality.  The  changes  in  carrier  rankings  (Table  1-3)  are  dramatic.  For 
example,  carrier  14  is  ranked  3rd  by  its  rates,  but  drops  to  l7th  when  its  quality  of 
past  performance  is  considered.  Similarly,  carrier  17  is  rated  16th  by  rates,  but 
moves  to  4th  because  of  its  high  quality  of  performance. 

We  then  measured  the  correlation  between  the  two  rankings  to  determine  the 
overall  effect  of  quality.  A  correlation  of  1  indicates  that  the  rankings  are  identical, 
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TABLE  1-2 


CARRIER  QUAirTT-ADJUSTED  RATES 


Carrier 

Average 

bid 

rate 

Rank 
by  bid 
rate* 

TQAP 

score 

Normalized 

quality 

score 

Adjusted 

quality 

rate 

Rank  by 
adjusted 
quality 
rate 

1 

.818 

20 

99.33 

1.064 

.769 

10 

2 

.768 

7 

99.40 

1.065 

.721 

3 

3 

.768 

8 

100.00 

1.071 

.717 

2 

4 

.802 

18 

92.87 

.995 

.806 

14 

S 

.768 

9 

97.48 

1.044 

.735 

5 

6 

.818 

21 

72.00 

.771 

1.060 

20 

7 

.768 

10 

95.11 

1.019 

.754 

8 

8 

.768 

11 

96.44 

1.033 

.743 

6 

9 

.736 

1 

91.09 

.976 

.754 

9 

10 

.768 

12 

95.50 

1.023 

.751 

7 

11 

.768 

13 

91.71 

.983 

.782 

12 

12 

.768 

14 

82.00 

.879 

.874 

18 

13 

.736 

2 

40.00 

.429 

1.717 

21 

14 

.736 

3 

82.00 

.879 

.838 

17 

15 

.736 

4 

84.00 

.900 

.818 

15 

16 

.736 

5 

98.00 

1.050 

.701 

1 

17 

.778 

16 

98.80 

1.059 

.735 

4 

18 

.802 

19 

94.88 

1.017 

.789 

13 

19 

.768 

15 

93.00 

.996 

.771 

11 

20 

.824 

22 

93.71 

1.004 

.821 

16 

21 

.736 

6 

82.00 

.879 

.838 

17 

22 

.786 

17 

73.33 

.786 

1.000 

19 

*  Carriers  with  the  seme  average  bid  rate  are  ranked  in  alphabetic  sequence 


while  a  correlation  of  0  (zero)  indicates  they  are  entirely  different.  We  found  a 
correlation  of  less  than  .02,  which  indicates  that  the  introduction  of  quality  markedly 
changed  the  carrier  rankings. 
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TABLE  1-3 


CARRIER  RANKING  BY  RATES  AMD  RATE/QUAU*nr  SCORES 


Carrier 

Rank  by  average 
bid  rate 

Rank  by  adjusted 
quality  rate 

9 

1 

9 

13 

2 

21 

14 

3 

17 

15 

4 

15 

16 

5 

1 

21 

6 

17 

2 

7 

3 

3 

8 

2 

5 

9 

5 

7 

10 

8 

8 

11 

6 

10 

12 

7 

11 

13 
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CONCLUSION 

The  results  of  our  preliminary  analysis  demonstrate  that  quality  could  have  a 
significant  effect  on  the  rankings  of  personal  property  carriers.  Low-rate  carriers 
with  poor-quality  performance  would  lose  some  of  their  cost-competitive  edge; 
carriers  offering  higher  rates  but  also  providing  high-quality  service  would  become 
more  competitive.  Clearly,  quality  does  make  a  difference. 


ORGANIZATION  OF  REPORT 


Building  on  the  preceding  results  and  MTMC’s  desire  to  award  personal 
property  shipments  on  the  basis  of  best  value,  the  balance  of  this  report  describes  an 
innovative  yet  practical  best-value  procurement  process.  We  begin  by  examining  a 
variety  of  U.S.  Government  and  private-sector  vendor  evaluation  programs;  identify 
three  alternatives  that  permit  MTMC  to  identify  best  value  when  procuring 
transportation  services;  present  a  best-value  operating  concept  for  the  preferred 
alternative;  and  conclude  with  guidelines  for  testing  and  evaluating  the 
recommended  concept. 
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CHAPTER 2 

BEST- VALUE  CONCEPTS  IN  INDUSTRY  AND  GOVERNMENT 


BACKGROUND 

Assessing  vendor  performance  and  awarding  business  contracts  on  a  best-value 
basis  enjoys  considerat'i  precedent  in  both  the  private  and  public  sectors.  Numerous 
organizations  have  designed  and  implemented  performance  evaluation  systems  for 
vendors  (suppliers)  of  both  goods  and  services.  Several,  including  GTE,  Kodak, 
Motorola,  Tennessee  Valley  Authority  (TVA),  Xerox,  and  the  General  Services 
Administration,  consider  carrier  performance  in  awarding  contracts  to  perform 
employee  relocations.  In  addition,  various  aerospace  companies  (e.g.,  Northrop  and 
Rockwell  International)  and  DoD  Components  (e.g..  Air  Force  Materiel  Command 
and  Defense  Logistics  Agency)  routinely  rate  the  performance  of  suppliers  from 
whom  they  purchase  parts,  materials,  and  services. 

Even  though  those  programs  focus  on  a  wide  variety  of  goods  and  services,  they 
share  a  number  of  common  objectives  and  characteristics,  which  we  examine  in  the 
next  two  sections. 

PROGRAM  OBJECTIVES 

Many  of  the  private  corporations  and  Federal  agencies  that  evaluate  their 
vendors  do  so  to  obtain  the  greatest  overall  value  in  the  goods  or  services  they 
procure.  Recognizing  that  poor  quality  is  expensive  and  destructive,  and  customers 
are  increasingly  demanding  satisfaction,  they  seek  to 

•  Identify  the  best  vendors 

•  Improve  the  quality  of  the  goods  and  services 

•  Reduce  the  costs  of  those  goods  and  services 

•  Expand  the  lines  of  communication  with  vendors. 

The  ultimate  objective  of  most  evaluation  programs,  regardless  of  their 
structure  or  design,  is  to  award  business  to  vendors  that  provide  the  best  value.  A 
secondary  objective  is  to  recognize  and  reward  the  higher  value  vendors  while 
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eliminating  those  unable  or  unwilling  to  meet  the  organization’s  standards. 
(Appendices  A  through  F  summarize  the  vendor  evaluation  programs  of  several 
organizations.) 

PROGRAM  CHARACTERISTICS 

Despite  the  variety  of  evaluation  programs,  they  are  generally 

•  Simple  to  administer,  they  avoid  the  use  of  complex  hardware,  software,  and 
procedures. 

®  Easy  to  understand-,  they  employ  self-explanatory  measures  and  reports. 

•  Automated-,  they  build  upon  existing  automation  capabilities  to  capture  and 
process  data. 

•  Based  on  available  information-,  they  use  data  that  are  routinely  collected  on 
carrier  performance. 

•  Timely  in  their  analysis-,  they  use  recent  data,  process  them  promptly,  and 
provide  timely  feedback  to  carriers. 

•  Oriented  toward  decisions-,  they  are  integrated  into  day-to-day  operations 
rather  than  serving  as  separate,  off-line  information  sources. 

•  Designed  to  foster  continuing  improvement-,  they  identify  and  share 
performance  trends  with  carriers  to  stimulate  improved  performance. 

•  Credible  and  legally  defensible;  they  employ  documented  and  supportable 
data. 

In  addition  to  these  characteristics,  various  changes  in  procurement  practices 
generally  accompany  the  more  successful  vendor  evaluation  programs.  They  include 
having  fewer  carriers  under  contract,  imposing  more  stringent  quality  standards  on 
carriers,  and  using  longer  contract  periods. 

POTENTIAL  METHODS 

This  blend  of  diverse  applications  and  common  characteristics  suggests  that 
MTMC  could  select  one  of  several  methods  for  determining  the  best  value  in 
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prociiriiig  personal  property  transportation  services.  Nonetheless,  the  method 
ultimately  adopted  should  enable  MTMC  to 

•  Identify  carriers  that  provide  the  best  value 

•  Award  shipments  according  to  best  value 

•  Measure  carrier  performance  on  a  continuing  basis 

•  Use  a  common  technical  methodology  that  is  applicable  to  all  situations. 

In  the  balance  of  this  section,  we  examine  three  different  alternatives  -  cost 
assessment,  member  survey,  and  indexing  —  that  MTMC  could  use  to  satisfy  these 
objectives. 

Cost  Assessment 

Synopsis 

Determine  the  cost  of  carriers  not  satisfying  selected  performance  factors  to 
obtain  best  value. 

Description 

In  this  method,  MTMC  would  determine  the  cost  of  various  performance  factors, 
such  as  late  pickup,  late  delivery,  shipment  turnback,  and  loss  or  damage.  Then,  for 
each  incident  of  inadequate  performance  during  a  specified  period,  it  would  assign 
those  costs  to  the  associated  personal  property  carriers.  The  total  of  those 
nonperformance  costs  and  the  carriers’  rates  for  specific  routes  would  be  the  best- 
value  rate,  the  basis  for  awarding  shipments. 

Assessment 

Although  this  method  translates  all  evaluation  factors  into  dollar  terms  and  is 
conceptually  simple,  determining  the  cost  of  each  evaluation  factor  (such  as  the  cost 
of  a  late  pickup)  would  he  difficult.  The  MTMC  could  determine  some  of  the  costs 
conducting  time  and  motion  studies  and  others  by  subjective  assessments.  In 
addition,  carriers  currently  submit  their  bid  rates  as  a  percent  of  MTMC’s  standard 
baseline  linehaul  transportation  rate  schedule.  The  actual  shipment  linehaul  costs 
are  not  known  until  a  carrier  submits  an  invoice  after  delivering  a  shipment.  The 
complexities  associated  with  determining  the  cost  of  various  quality  performance 
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factors  and  translating  bid  rates  into  dollars  would  make  this  method  very  difficult 
for  MTMC  to  implement  and  maintain. 

Member  Survey 

Synopsis 

Use  member  responses  to  survey  questions  to  establish  the  level  of  carrier 
performance. 

Description 

In  this  method,  MTMC  would  request  that  every  member  complete  a  personal 
property  carrier  performance  survey  form  and  submit  it  to  a  PPSO  following 
shipment  delivery.  Although  MTMC  could  structure  and  score  the  responses  to  such 
a  survey  in  many  different  ways,  one  alternative  would  be  to  assign  a  numerical 
value  to  a  particular  level  of  satisfaction  for  every  evaluation  factor.  For  example,  in 
response  to  a  question  regarding  the  cleanliness  of  the  moving  van’s  interior,  it  could 
offer  four  levels  of  satisfaction:  excellent  =  4;  satisfactory  =  3;  fair  =  2;  and 
unsatisfactory  =  1. 

Assessment 

Private-sector  companies  use  this  technique  extensively,  but  they  relocate 
relatively  few  employees.  The  major  advantage  of  this  technique  is  that  it  uses  the 
customers  (i.e.,  the  individuals  moved)  rather  than  a  third  party  to  evaluate  the 
carriers.  In  the  DoD  environment,  however,  customer  feedback  may  not  be  received 
in  a  timely  manner,  if  at  all.  In  addition,  the  evaluations  would  be  subjective,  the 
process  would  be  administratively  burdensome  because  of  the  program’s  size,  and  the 
translations  of  the  members’  quality  evaluations  into  rate  adjustments  would  be 
difficult. 

Indexing 

Synopsis 

Develop  either  an  adjusted  rate  or  performance  score  for  each  carrier  based 
upon  bid  rates  and  demonstrated  levels  of  quality  performance. 
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Description 

In  this  method,  MTMC  would  use  a  mathematical  formula  to  compute  an  index, 
expressed  as  either  a  Best  Value  Rate  or  Best  Value  Score.  It  would  then  use  that 
index  to  rank  the  carriers  on  the  appropriate  TDRs.  The  formula  would  include  the 
carrier’s  bid  rate  and  selected  quality  performance  factors. 

Assessment 

These  types  of  evaluation  processes  are  well-tested,  objective,  and  adaptable  to 
personal  property  transportation  services.  Because  the  proposed  application  would 
take  into  account  both  a  carrier’s  bid  rate  and  selected  quality  performance  factors  to 
define  overall  performance,  MTMC  should  apply  the  formula  using  a  standard  unit  of 
measure  for  the  factors  (such  as  percentages). 

Use  of  either  a  Best  Value  Rate  or  a  Best  Value  Score  would  enable  MTMC  to 
rank  each  personal  property  carrier  according  to  the  overall  value  it  offers  the  DoD. 

CONCLUSION 

We  believe  that  using  indexing  methods  will  enable  MTMC  to  effectively  link 
carriers’  quality  performance  to  their  bid  rates  in  order  to  determine  best  value. 
Although  MTMC  could  use  various  ways  to  structure  and  apply  an  index,  two  -  Best 
Value  Rate  and  Best  Value  Score  —  show  particular  promise.  Both  identify  best 
value,  are  easy  to  use  and  administer,  and  provide  tbe  flexibility  necessary  to  change 
and  upgrade  MTMC’s  personal  property  transportation  service  procurement  process. 

In  the  next  chapter,  we  present  alternative  technical  approaches  that  MTMC 
could  use  to  guide  its  application  of  these  two  indexing  methods. 
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CHAPTERS 


APPLYING  BEST- VALUE  TECHNIQUES 
TO  PERSONAL  PROPERTY  PROCUREMENT 


Both  of  the  proposed  indexing  methods  described  in  the  previous  chapter  are 
designed  to  identify  carriers  that  provide  the  greatest  overall  value  to  the  DoD.  The 
Best  Value  Rate  identifies  a  relative  level  of  carrier  quality  performance,  while  the 
Best  Value  Score  provides  a  composite  of  the  carrier’s  rate  and  quality  performance. 
Regardless  of  the  indexing  method  selected,  MTMC  should  compute  a  quality  index 
semiannually  for  each  carrier.  To  compute  either  index,  MTMC  needs  to 

•  Select  the  most  important  quality  performance  factors 

•  Develop  the  relative  degree  of  importance  (weight)  of  those  performance 
factors 

•  Use  a  standard  methodology  or  formula  to  compute  indexes. 

We  describe  each  of  these  steps  in  more  detail  below. 

SELEaiNG  QUALITY  PERFORMANCE  FAaORS 

A  normal  shipment  cycle  encompasses  a  series  of  many  events,  including  the 
award  of  a  shipment  to  a  carrier,  delivery  of  the  shipment  to  final  destination,  and 
payment  for  the  services  provided.  Although  MTMC  could  determine  the  quality  of 
performance  of  each  of  those  events  and  many  more,  we  believe  that  is  unnecessary. 
Drawing  upon  the  experience  of  both  public-  and  private-sector  applications  of  vendor 
evaluation  programs,  MTMC  should  focus  on  the  rate  a  carrier  bids  and  just  four 
performance  factors  —  timeliness  of  pickup,  timeliness  of  delivery,  number  of  loss  or 
damage  claims,  and  number  of  turnbacks  or  pullbacks.  It  should  develop  measures 
for  those  five  factors  using  the  carrier’s  performance  during  the  most  recently 
completed  6-month  rate  cycle.  Furthermore,  it  should  normalize  those  measures  by 
expressing  them  in  terms  of  the  percentage  of  the  total  number  of  shipments  that 
were 


•  Picked-up  on  time 

•  Delivered  on  time  (met  the  required  delivery  date) 
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•  Free  of  loss  and/or  damage 

•  Free  of  turnback  (by  the  carrier)  or  pullback  (by  the  PPSO). 

The  data  for  each  of  these  five  factors  are  currently  available  from  MTMC’s 
automated  personal  property  systems. 

DEVELOPING  WEIGHTS 

A  critical  f  of  any  index  is  ensuring  that  each  factor  comprising  the  index 
is  fairly  and  a*  /sighted.  The  MTMC  could  use  a  variety  of  techniques  for 

determining  those  ..  .  j,  but  three  are  particularly  applicable: 

•  Statistical  —  assign  weights  through  the  use  of  principal  component 
analysis 

•  Subjective  -  use  some  type  of  “expert  choice”  technique2 

•  Subjective  -  use  the  opinion  of  one  or  more  recognized  experts. 

Table  3-1  summarizes  the  advantages  and  disadvantages  of  each  of  the  three 
techniques. 


TABLE  M 

ALTERNATIVE  WEIGHTING  TECHNIQUES 


Technique 

Advantage 

Disadvantage 

Statistical 

Subjective  (Expert  choice) 
Subjective  (Expert  opinion) 

Objective,  supp  irtable 
Consistent,  logical 

Quick,  easy  to  use 

Requires  statistical  expertise 
Somewhat  subjective 

Highly  subjective 

1  Principal  component  analysis  is  particularly  useful  for  developing  indexes  that  are  based  on 
the  interdependence  of  the  variables.  For  the  mathematical  treatment  of  principal  component/^,  see 
M.  G.  Kendall  and  A.  Stuart,  The  Advanced  Theory  of  Statistics,  Volume  3,  London:  Griffin,  1966. 

2Exp?rt  choice  is  an  automated  decision  support  process  that  provides  a  systematic  approach  to 
organizing  problems  in  order  to  reach  informed,  logical  decisions.  It  portrays  the  relationships  among 
all  facets  of  a  complex  problem  and  uses  quantitative  and  qualitative  information,  to  include 
experience  and  intuition. 
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Although  MTMC  could  determine  the  weights  using  expert  choice  or  expert 
opinion,  we  believe  determining  the  weights  statistically  is  the  most  objective 
method. 

To  determine  the  best-value  index  statistically  (by  principal  component 
analysis),  MTMC  needs  to 

•  Transform  the  different  best-value  factors  -  e.g.,  on-time  pickup,  on-time 
delivery,  and  damage-free  movement  -  into  standard,  normal  variables  by 
first  expressing  them  as  deviations  from  their  means  and  then  dividing  by 
their  respective  standard  deviations.  These  steps  convert  each  of  the  best- 
value  factors  into  a  pure  number  with  a  mean  of  zero  and  variance  equal 
to  1. 

•  Assign  weights  to  the  standard,  normal  variables  so  that  their  inherent 
variances  are  reflected  as  much  as  possible  in  the  resultant  composite  best- 
value  index.  To  obtain  a  solution,  constrain  the  square  of  the  weights  in  the 
composite  measure  so  that  they  sum  to  unity. 

These  steps  can  be  readily  accomplished  on  a  personal  computer  with  commercial 
software. 

COMPUTING  INDEXES 

This  section  presents  a  technical  approach  for  each  of  the  two  indexing  methods 
and  then  compares  some  of  the  characteristics  of  the  two  methods. 

Best  Value  Rates 

The  Best  Value  Rate  method  adjusts  each  carrier’s  bid  rate  based  upon  the 
carrier’s  performance  in  the  four  quality  areas  described  previously.  Once  M'TMC 
has  determined  the  weight  for  each  of  the  quality  performance  factors  (bid  rates  are 
not  a  weighted  evaluation  factor  in  this  method),  it  then  computes  a  Best  Value  Rate 
for  each  carrier,  by  route,  at  each  origin  PPSO. 

This  method  employs  an  index  to  determine  a  carrier’s  Quality  Performance 
Score  and  to  compute  a  carrier's  Relative  Quality  Score. 3 

The  MTMC  would  use  the  Relative  Quality  Score  to  adjust  the  carrier’s  bid  rate 
up  or  down  —  up  if  the  Relative  Quality  Score  is  low,  and  down  if  it  is  high.  If  the 


■■^The  term  “relative"  in  this  expression  means  the  carrier’s  performance  in  relation  to  the 
average  performance  of  all  other  carriers  serving  the  PPSO. 


quality  performance  of  the  carrier  being  evaluated  is  equal  to  the  average 
performance  of  all  carriers  serving  the  PPSO,  then  its  Relative  Quality  Score  would 
be  unity  (1),  and  the  carrier’s  bid  rate  would  remain  unchanged. 

Simple  arithmetic  can  then  be  used  to  convert  each  carrier’s  bid  rate  to  a  Best 
Value  Rate.  The  process  consists  of  four  steps. 

•  Step  1:  Compute  a  Quality  Performance  Score  using  the  four  weighted 
quality  performance  factors. 

Quality  Performance  Score  =Wi(On-Time  Pickup  Percent)  +  W2(On-Tim€  Delivery 
Percent)  +  W^lLoss-ZDamage-Free  Percent)  +  W^lTurnback'/Pullback-Free 
Percent),  where  the  W,  are  the  relative  weights. 

•  Step  2:  Compute  a  Carrier  Average  Performance  Score  using  the  Quality 
Performance  Scores  of  all  carriers  serving  the  origin  PPSO. 

•  Step  3:  Divide  the  carrier’s  Quality  Performance  Score  by  the  Carrier 
Average  Performance  Score  to  produce  a  Relative  Quality  Score. 

Relative  Quality  Score =Quality  Performance  Score  Carrier  Average  Performance 

Score 

•  Step  4;  Divide  the  carrier’s  bid  rate  by  the  Relative  Quality  Score  to  produce 
a  Best  Value  Rate  (the  bid  rate  adjusted  for  quality). 

Best  Value  Rate —Bid  Rate  +  Relative  Quality  Score. 

The  Best  Value  Rates  would  then  permit  MTMC  to  realign  carriers  on  each 

TDR. 

Best  Value  Scores 

Although  this  indexing  method  also  uses  simple  arithmetic,  it  differs  from  the 
above  computation  in  one  important  aspect:  it  assigns  a  weight  to  the  carrier’s  bid 
rate  in  the  formula. 

Once  MTMC  has  determined  the  weights  for  the  five  evaluation  factors,  it  is 
ready  to  compute  a  Best  Value  Score  for  each  carrier,  by  route,  at  each  origin  PPSO, 
using  the  following  formula: 

Best  Value  Score -Vf j( Rate  Bid  Percent)  +  W^CO/i-Time  Pickup  Percent)  +  Vf^tOn-Time 
Delivery  Percent)  +  W4(Loss-/Damage-Free  Percent)  +  Ws(Turnback-/Pullback. Free  Percent), 
where  the  W,  are  the  relative  weights. 
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In  addition  to  expressing  all  units  of  measure  as  a  percentage,  two  additional 
rules  for  ensuring  consistency  in  the  Best  Value  Score  formula  are  as  follows: 

•  Rule  1  -  Each  of  the  five  factors  comprising  the  formula  (the  carrier’s  rate 
and  four  quality  factors)  is  based  on  a  100  percent  maximum. 

•  Rule  2  -  Higher  is  better. 

The  four  quality  factors  in  the  formula  are  straightforward.  For  example,  if  a 
carrier  picks  up  97  percent  of  its  shipments  on  time,  then  MTMC  should  use 
97  percent  in  the  formula.  However,  the  rate  factor  is  shown  in  one  of  three  ways; 

•  A  positive  percentage,  when  the  bid  is  less  than  the  MTMC  rate  schedule 

•  A  zero,  when  the  bid  is  equal  to  the  MTMC  rate  schedule 

•  A  negative  percentage,  when  the  bid  is  greater  than  the  MTMC  rate 
schedule. 

The  Best  Value  Scores  would  then  enable  MTMC  to  realign  carriers  on  each 
TDR  accordingly. 

Comparing  the  Indexing  Methods 

Both  of  the  proposed  indexing  methods  link  a  carrier’s  rate  to  the  quality  of  its 
performance.  The  Best  Value  Score,  however,  has  two  primary  advantages: 

•  It  is  more  direct;  it  requires  only  one  computation  as  opposed  to  four  for  Best 
Value  Rate. 

•  It  does  not  assume  a  direct  relationship  between  quality  costs  and  bid  rates; 
the  Best  Value  Rate  imputes  different  quality  costs  to  carriers  with  the  same 
quality  record.  The  result  is  that  carriers  may  not  be  treated  equally  when 
adjusting  bid  rates  for  quality  by  the  Best  Value  Rate  procedure. 4 


^This  phenomenon  is  illustrated  in  the  following  table; 


Carrier 

Bid  rate  (%  of 
MTMC  standard) 

Relative  carrier 
quality  (%) 

Best-value  rate 

Implied  quality  cost 
(best-value  rate 
minus  bid  rate) 

A 

70 

70 

100 

30 

B 

98 

70 

140 

42 

Nott:  Carriers  A  arid  6  have  the  same  quality  record,  which  is  70  percent  as  good  as  the  industry.  Yet,  when 
applied  against  the  bid  rate,  that  same  quality  record  «  treated  differently;  higher  for  Carrier  B  (42  percentage 
points)  and  lower  for  Carrier  A  (30  percentage  points).  In  general,  the  cost  of  a  late  delivery  should  be  the  tame  for 
both  carriers  and  not  higher  for  Carrier  B  simply  because  it  bid  higher  (98  percent)  than  Carrier  A  (70  percent) 
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We  believe  these  advantages  provide  a  strong  incentive  for  MTMC  to  use  the 
Best  Value  Score  method.  Table  3-2  shows  how  a  hypothetical  application  of  the  Best 
Value  Score  impacts  the  ranking  of  carriers  for  procurement  purposes.  The  example 
is  hypothetical  because  we  have  estimated  the  weights  against  which  the  five 
evaluation  factors  are  applied.  (The  actual  weights  will  have  to  be  derived  if  MTMC 
chooses  to  test  or  adopt  the  Best  Value  Score  concept.)  Calculating  Best  Value  Scores 
for  Carriers  A,  B,  and  C  is  shown  below  (other  carriers  are  calculated  similarly): 


Best  Value  Score  A  =  .39(41)  +  .07(73)  +  .18(79)  +  .26(60)  +  .10(99)  =  60.82. 

Best  Value  Score  B  =  .39(41 )  +  .07(91)  +  .18(70)  +  .26(48)  + .  10(93)  =  56.74. 

Best  Value  Score  C=  .39(41)  +  .07(54)  +  .18(73)  +  .26(63)  +  .10(100)  =  59.29. 

In  these  calculations,  the  sequence  of  formula  factors  is  bid-rate  percent,  on-time 
pickup  percent,  on-time  delivery  percent,  loss-/damage-free  percent,  and  turnback-/ 
pullback-free  percent.  Carrier  bid  rates  are  stated  in  terms  of  a  percent  above  or 
below  the  MTMC  baseline  linehaul  standard  (100  percent).  Therefore,  a  bid  is 
recorded  as  the  percent  of  difference  between  100  and  the  bid  (in  the  above  example, 
each  of  the  three  carriers  bid  59  percent,  which  is  entered  into  the  formula  as 
41  percent).  A  hid  above  the  standard  is  recorded  as  a  negative  percent  (for  example, 
a  carrier  bid  of  130  percent  is  shown  as  —  30  percent). 

It  can  be  seen  from  Table  3-2  that  the  Best  Value  Score  method  does  result  in  a 
reordering  of  carriers.  Those  that  perform  better  move  up  in  rank.  That  reordering, 
however,  presents  a  challenge  in  allocating  traffic  among  the  best-value  carriers. 
That  challenge  is  discussed  next. 

ALLOCATING  TRAFFIC  TC  ARRIERS 

Our  focus  thus  far  has  been  on  identifying  and  describing  a  best-value 
methodology.  However,  the  procedures  MTMC  uses  to  allocate  traffic  among  best- 
value  carriers  are  as  important  as  the  techniques  used  to  determine  best  value. 

Under  the  PPTMR’s  current  shipment  allocation  procedures,  MTMC  awards 
shipments  equitably,  by  weight,  to  carriers  within  each  r;  ’'oup.  However, 
indexing  methods  result  in  a  revised  TDK  that  lists  carriers  basen  on  the  highest  to 
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TABLE  3-2 


CARRIER  RANKING  BY  BIO  RATE  AND  BEST  VALUE  SCORE 


Carritr 

Bid  rate 
(weight 
.39%) 

Bid  rate 
rank 

OTP 

(weight 

.07%) 

OTO 

(weight 

.18%) 

L/DF 

(weight 

.26%) 

T/PF 

(weight 

.10%) 

BVS 

BVS 

rank 

A 

59 

1 

73 

79 

60 

99 

60.82 

2 

B 

59 

91 

70 

48 

93 

56.74 

6 

C 

59 

(41) 

54 

73 

63 

100 

59.29 

4 

D 

65 

(35) 

99 

82 

70 

100 

63.54 

1 

E 

65 

(35) 

5 

100 

89 

58 

88 

60.55 

3 

83 

(17) 

6 

58 

75 

75 

95 

53.19 

11 

88 

(12) 

7 

75 

100 

63 

94 

53.71 

10 

H 

90 

(10) 

8 

43 

90 

87 

89 

54.63 

8 

1 

90 

(10) 

9 

100 

83 

91 

79 

57.40 

5 

J 

90 

(10) 

10 

81 

100 

55 

99 

51.7/ 

13 

mm 

100 

(0) 

11 

92 

82 

88 

100 

54.08 

9 

MM 

100 

(0) 

12 

64 

88 

90 

93 

53.02 

12 

M 

100 

(0) 

13 

87 

93 

87 

97 

55.21 

■■ 

N 

110 

(-10) 

14 

100 

94 

83 

96 

51.20 

MM 

120 

(-20) 

15 

83 

91 

100 

95 

49.89 

19 

H 

120 

(-20) 

16 

56 

82 

90 

92 

43.48 

H 

Nom:  OTPsen-timc  pickup;  OTD  =  on-time  dtlivtry;  L/OF>l(Kt/d«m«g«  free;  T/PFs turnback/pullback  frac:  and 
6VS  K  Baft  Valua  Scora. 


lowest  best>value  rates  or  scores  —  efTectively  doing  away  with  rate  groups.  Thus,  a 
new  scheme  for  allocating  shipments  needs  to  be  devised.  We  see  three  alternatives. 

The  first  option  is  to  award  all  traffic  to  the  single  best- value  carrier.  In  our 
judgment  this  is  not  practical  because,  in  most  situations,  a  single  carrier  is  not 
capable  of  absorbing  this  volume  of  traffic.  Further,  performance  data  for  excluded 
carriers  would  quickly  become  dated,  making  it  difficult  to  determine  their  Best 
Value  Rates  or  Scores. 

Under  the  second  option,  MTMC  would  recognize  the  rate  groups  and  continue 
to  award  shipments  evenly  within  them.  Consequently,  the  Best  Value  Rates  or 
Scores  would  be  used  only  to  determine  which  carrier  gets  the  first  shipment.  This 
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option  would  be  easy  to  implement  and  administer,  for  it  continues  current  practices. 
However,  the  practice  of  awarding  approximately  equal  shipment  weights  to  all 
carriers  within  the  same  rate  group  tends  to  defeat  the  best-value  concept  and  fails  to 
provide  the  necessary  incentives  for  quality  performance. 

The  third  option,  Best  Value  Clusters,  would  call  for  MTMC  to  award  shipments 
to  carriers  clustered  by  Best  Value  Rates  or  Best  Value  Scores.  This  practice  would 
give  all  carriers  an  incentive.  It  also  would  maintain  a  degree  of  equitable 
distribution  and  may  therefore  '■•e  easier  for  carriers  to  accept.  To  illustrate  how 
MTMC  might  cluster  its  carriers  according  to  Best  Value  Scores,  Table  3-3  shows  a 
proposed  clustering  for  a  route  serviced  by  50  carriers.  The  carrier  identification 
numbers  correspond  to  the  Best  Value  Scores  arranged  from  highest  to  lowest  (i.e.. 
Carrier  1  has  the  highest  Best  Value  Score  and  Carrier  50  has  the  lowest).  Each 
group  of  carriers  is  considered  a  cluster.  In  this  scheme,  MTMC  would  allocate 
50  percent  of  the  shipments  to  the  top  10  carriers,  30  percent  to  the  next  10  carriers, 
etc.  Each  cluster  should  contain  an  equal  number  of  carriers  except  for  the  last 
cluster,  which  should  contain  all  of  the  remaining  carriers.  This  ensures  that  no 
carrier  in  a  lower  cluster  gets  more  traffic  than  any  carrier  in  a  higher  cluster. 

TABLE  3-3 

POTENTIAL  APPLICATION  OF  BEST  VALUE  CLUSTERS 


Cluster 

Carrier 

identification 

number 

Allocation  (%) 

1 

1  -  10 

50 

2 

n  -  20 

30 

3 

21  -  30 

15 

4« 

31  -  50 

5 

•  All  r«mainin9  carriers 


Under  this  cluster  concept,  MTMC  could  award  shipments  equally  (by  weight) 
among  all  carriers  in  a  cluster  or,  alternatively,  a  disproportionate  share  to  the  top 
carriers  (e.g.,  the  top  carrier  in  the  top  cluster  would  receive  50  percent  of  the  traffic 
set  aside  for  that  cluster). 
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Either  way,  we  believe  how  MTMC  allocates  traffic  among  its  best- value 
carriers  is  as  important  as  how  it  calculates  best  value.  Because  of  the  variety  of 
alternatives  in  both  calculating  best  value  and  allocating  traffic,  we  believe  it  is  in 
MTMC’s  best  interest  to  test,  on  a  small  scale,  the  various  options  at  one  or  more 
sites.  Those  combinations  of  alternative  approaches  that  prove  to  be  most  effective 
can  then  be  expanded  throughout  the  DoD. 

OTHER  CONCERNS 

A  best -value  indexing  method,  like  any  new  and  untested  procedure,  raises 
some  concerns  that  MTMC  needs  to  resolve  before  it  moves  forward  with 
implementation.  We  present  three  of  those  concerns  in  this  section,  along  with  some 
possible  solutions. 

One  concern  is  how  MTMC  would  integrate  new  carriers  into  the  process.  Any 
new  carrier  accepted  for  shipment  award  during  the  evaluation  period  will  not  have  a 
shipment  record  that  MTMC  could  use  to  compute  either  a  Best  Value  Rate  or  Best 
Value  Score.  For  those  carriers,  MTMC  could  assign  an  average  Best  Value  Rate  or 
Score,  with  the  average  computed  from  either  historical  data  or  the  previous 
evaluation  cycle  data.  New  carriers  would  still  need  to  submit  low  bids  to  receive  any 
shipments,  however. 

A  second  concern  is  how  MTMC  would  treat  carriers  that  were  assigned  to  a 
shipment  allocation  cluster  but  received  no  shipments  during  the  evaluation  period 
(resulting  in  MTMC  being  unable  to  compute  a  Best  Value  Rate  or  Score  based  on 
demonstrated  quality  performance).  This  is  a  common  occurrence  today.  Again,  for 
these  carriers,  MTMC  could  use  the  average  of  their  Best  Value  Rates  or  Scores  over 
all  the  other  routes  for  which  shipment  awards  were  made  during  that  period. 

The  third  concern  is  how  MTMC  would  treat  high-  and  low-volume  carriers. 
The  MTMC  can  expect  to  be  challenged  by  any  carrier  awarded  only  a  few  shipments 
if  that  carrier  is  clustered  with  others  that  received  many  shipments.  To  avoid  such 
challenges,  MTMC  could  designate  volume  ranges  (for  example,  low,  medium,  and 
high)  and  categorize  carriers  accordingly  when  computing  each  carrier’s  Best  Value 
Rate  or  Score.  That  solution  would  also  force  MTMC  to  develop  a  separate  weighting 
scheme  for  each  range. 
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SUMMARY 


In  this  chapter,  we  examined  various  DoD  personal  property  transportation 
program  practices  that  are  key  for  structuring  a  best-value  concept.  We  also 
described  two  alternative  indexing  approaches  for  determining  best-value  carriers, 
proposed  alternative  methods  for  allocating  shipments  to  those  carriers,  and 
identified  some  concerns  and  potential  solutions.  Because  of  the  complexities  of 
initiating  and  sustaining  a  best-value  procurement  program,  we  believe  it  is  in 
MTMC’s  best  interest  to  test,  on  a  small  scale,  those  techniques  and  alternative 
approaches  outlined  in  this  report.  Tlte  next  chapter  provides  details  on  such  a  test. 
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CHAPTER  4 


TESTING  BEST-VALUE  PROCUREMENT  PRACTICES 


Both  the  Best  Value  Rate  and  Best  Value  Score  methods  should  permit  PPSOs 
to  award  personal  property  shipments  to  carriers  based  upon  the  highest  overall 
value  considering  both  linehaul  rates  and  performance  quality.  Nonetheless,  MTMC 
should  exercise  caution  as  it  moves  forward  with  the  concept  of  best  value  in  personal 
property  transportation  service  procurement.  It  already  receives  some  increase  in 
“value"  through  TQAP,  which  requires  carriers  to  attain  a  minimum  quality  score  to 
remain  eligible  for  shipment  award.  As  a  consequence,  any  best-value  method  that 
MTMC  adopts  must  result  in  a  substantial  improvement  over  current  practices.  To 
demonstrate  the  potential  of  best  value,  we  recommend  that  MTMC  test  it  in  an 
operating  environment.  In  this  chapter,  we  propose  a  plan  for  such  a  test. 

OVERVIEW 

In  keeping  with  MTMC's  desire  to  develop  a  best-value  concept  using  only 
Code  1  (door-to-door  motor  van)  transportation  services,  we  propose  that  it  conduct 
the  test  using  actual  shipment  data.  The  specific  objectives  of  the  test  should  include 
the  following; 

•  Select  a  best-value  indexing  method 

•  Develop  a  user-friendly  computer  program  to  support  best-value 
procurement 

•  Adopt  enhanced  shipment  allocation  practices 

•  Decide  whether  best-value  procurement  satisfies  DoD’s  requirements. 

At  a  minimum,  we  further  propose  that  MTMC  structure  its  test  to  accom¬ 
modate  the  procedures  described  below. 

•  Test  alternative  indexing  methods  by  computing  both  a  Best  Value  Rate  and 
a  Best  Value  Score  for  all  carriers  serving  selected  test  routes,  at  two  or  more 
PPSOs.  These  PPSOs  should  encompass  more  than  one  Military  Service, 
represent  vari^'d  v/orkloads,  and  have  TOPS  capability.  We  further  suggest 
that  MTMC  coordinate  with  the  Military  Services  to  identify  appropriate 
test  sites  and  that  it  prepare  and  publish  a  test  plan  that  lays  out  the 
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objectives  of  the  test  and  the  responsibilities  and  requirements  of  the  test 
participants. 

•  Test  alternative  carrier  selection  and  traffic  allocation  practices  at  each  of 
the  test  sites. 

•  Apply  the  recommended  carrier  selection  and  shipment  allocation 
procedures  to  the  Best  Value  Rates  and  Best  Value  Scores  to  determine 
which  combination  of  alternatives  is  most  practical  and  worthwhile. 

We  estimate  that  8  months  should  be  a  sufficient  period  to  conduct  a  thorough 
test.  Figure  4-1  presents  the  major  milestones  and  proposed  timelines  for  such  a  test. 


Milestone 


Coordinate  test  plan  with  Military 
Services 

Selea  PPSOs  to  serve  as  test  sites 

Selec.  the  start/end  dates  of  the  carrier 
performance  evaluation  period 

Collect  carrier  rate  and  quality 
performance  data 

Determine  weighting  scheme  for  each 
indexing  method 

Calculate  a  Best  Value  Rate  and  Score 
for  each  carrier 

Paper-test  traffic  award  and  allocation 
alternatives 

Conduct  test 
Analyze  test  results 
Recommend  a  best-value  process 

Detail  a  best  value  implementation 
plan  and  resource  requirements 


1 


Month 


D 

n 

a 

1 

■ 

FIG.  4-1.  BEST  VALUE  TEST  MILESTONES  AND  SCHEDULE 


SUMMARY 


Industry  is  using  various  techniques  to  assure  best  value  in  transportation 
procurement.  Many  of  those  techniques  hold  promise  for  the  DoD.  However,  faced 
with  alternative  indexing  and  traffic  allocation  opti-^ns,  MTMC  needs  to  demonstrate 


which  combinations  offer  the  most  practicality,  utility,  and  cost-effectiveness.  We 
believe  the  product  of  that  demonstration  test  is  likely  to  be  a  model  for  the  procure¬ 
ment  of  all  transportation  services  in  the  future. 
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APPENDIX  A 

GTE  SERVICE  CORPORATION 


TITLE 


Household  Goods  Quality  Evaluation  Program 

PROGRAM  DESCRIPTION 

In  response  to  the  financial  cost  and  emotional  nature  of  moving  employees’ 
household  goods,  GTE  developed  in  1987  a  program  to  measure  and  enhance  the 
quality  of  service  provided  by  its  contracted  household  goods  carriers.  GTE  relocates 
3,500  personnel  annually,  at  a  total  cost  of  approximately  $12  million,  under  3-year 
contracts  with  four  household  goods  carriers. 

The  primary  features  of  GTE’s  household  goods  movement  program  include  the 
following: 

•  Centralized  bids  from  carriers  with  decentralized  (by  region)  booking  of 
shipments  with  carriers 

•  No  tonnage  or  shipment  guarantees  to  carriers 

•  Carriers  selected  by  a  committee. 

That  program  makes  extensive  use  of  quality  standards  and  supporting  systems 
to  measure  the  level  of  service  provided  by  its  household  goods  carriers.  It  currently 
calls  for 

•  Six-month  evaluation  cycles 

•  Completion  of  a  quality  rating  form  by  each  relocating  employee 

•  Stringent  quality  standards  i 

►  On-time  pickup,  99  percent 

►  On-time  delivery,  99  percent 


IGTE  Household  Goods  Quality  Evaluation,  28  April  28  1992. 
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>  Frequency  of  loss  or  damage,  12  percent  or  less 
»  Amount  of  loss  or  damage,  $300  or  less 

^  Pre-move  survey  performed,  100  percent 
i>  Overall  service  rating,  90  percent  “excellent/good” 

>  Overall  service  rating,  2  percent  or  less  “poor” 

>  GTE  personnel  rating,  95  percent  “would  use  again.” 

The  core  of  GTE’s  program  is  the  shipment  evaluations  provided  by  relocating 
employees.  Those  evaluations  determine  the  share  of  GTE’s  business  that  each 
carrier  receives  and  how  much  it  is  paid  for  each  shipment.  In  early  1989,  GTE  began 
an  incentive  pay  system  under  which  it  pays  carriers  a  percentage  above  or  below  the 
contracted  rate  depending  upon  the  results  of  the  employee  evaluations:  shipments 
rated  excellent  are  paid  at  108  percent;  good  at  105  percent;  and  poor  at  90  percent. 
In  addition,  carriers  that  meet  or  exceed  specified  quality  standards  receive  annual 
rate  adjustments. 

PROGRAM  RESULTS 

The  first  household  goods  carrier  evaluations  in  1987  revealed  a  variety  of 
quality  problems:  92  percent  on-time  pickup,  89  percent  on-time  delivery,  and 
35  percent  frequency  of  loss  and  damage  claims.  In  addition,  only  76  percent  of  GTE 
employees  noted  they  would  use  the  same  carrier  in  a  subsequent  move.2 

When  the  evaluation  results  and  relative  carrier  rankings  were  provided  to 
relocating  employees,  who  have  the  option  of  selecting  a  carrier  from  GTE’s  approved 
list,  the  carrier  with  the  highest  quality  rating  immediately  doubled  its  share  of 
GTE’s  business.  Similarly,  the  carrier  with  the  lowest  quality  rating  saw  a 
significant  decrease  in  its  share  of  GTE  business. 

Although  the  program  quickly  led  to  an  increase  in  the  quality  of  carrier 
performance,  the  carriers  were  still  not  routinely  meeting  GTE’s  standards 
18  months  later.  That  triggered  the  1989  incentive  pay  program.  The  associated 


21nformation  in  this  paragraph  was  taken  primarily  from  DISTRIBUTION  magazine,  “Pay  for 
Performance  Boosts  HHG  Service,"  pages  64-68,  November  1990. 


18  percent  spread  (from  90  percent  to  108  percent)  in  revenue  provided  the  necessary 
incentive  for  the  carriers  to  make  further  improvements  in  performance. 

•  Overall  service  quality  and  ratings  of  “would  use  again”  jumped  to 
90  percent. 

•  Damage  frequency  dropped  to  1 1  percent,  exceeding  the  GTE  standard. 

This  emphasis  on  quality  clearly  changed  the  focus  of  GTE’s  household  goods 
movement  program  from  lowest  transportation  cost  to  quality  service  at  lowest 
overall  cost.  The  results  are  manifested  both  in  more  satisfied  employees  and 
improved  financial  benefits  to  GTE  and  the  carriers. 


APPENDIX  B 

EASTMAN  KODAK  COMPANY 


TITLE 


Performance  Measurements/Excellence  Award  Program 

PROGRAM  DESCRIPTION 

The  Eastman  Kodak  Company  measures  the  performance  of  carriers  that  it 
uses  to  relocate  employees  to  recognize  excellent  service  and  to  grant  national  and 
regional  awards  to  the  best  carriers.  In  1992,  Kodak  contracted  with  four  carriers 
and  two  agents  to  relocate  approximately  2,300  employees. 

Kodak  makes  shipment  awards  to  carriers  only  on  the  basis  of  quality, 
primarily  because  the  carriers’  rates  are  very  similar.  In  assessing  the  quality  of 
carrier  performance,  Kodak  considers  three  operational  factors,  two  administrative 
factors,  and  customer  surveys.  Table  B-1  shows  the  relationships  among  those 
factors. 


TABLE  B-1 

KODAK'S  CARRIER  PERFORMANCE  MEASURES 


Performance  categories 

Operational 

performance 

(50%) 

Administrative 

performance 

(30%) 

Customer 

survey 

(20%) 

Factor 

Maximum 

point 

value 

Factor 

Maximum 

point 

value 

Factor 

i  1 

Maximum 

point 

value 

On-time  service 

25 

Shipment  registration 

n 

Customer  rating 

20 

Claims  incidents 

15 

Document  scanning 

Claims  value 

10 

■■ 

Operational  Performance 

Kodak  evaluates  on-time  service  using  a  basis  of  25  points,  with  100  percent  on- 
time  service  earning  25  points  and  performance  at  less  than  95  percent  earning 
0  points.  It  also  monitors  the  number  of  claims  filed  as  a  percentage  of  total 
shipments  (claims  incidents),  awarding  carriers  15  points  for  no  claims  and  0  points 
for  performance  at  1.51  percent  or  greater.  The  dollar  value  of  claims  as  a  percentage 
of  revenue  (claims  value)  is  another  important  factor.  Kodak  awards  carriers 
10  points  for  no  claims  and  0  points  for  a  claims  value  percentage  of  1.51  or  greeter. 

Administrative  Performance 

Accurate  shipment  registration  reflects  the  number  of  registration  errors  as  a 
percentage  of  the  total  number  of  shipments  registered.  Kodak  measures  the 
carrier’s  performance  in  registering  shipments  using  a  scale  that  ranges  from 
15  points  for  100  percent  performance  to  0  points  for  performance  below  95  percent. 
Kodak  is  also  concerned  with  the  carrier’s  accuracy  in  submitting  documents  for 
automated  scanning.  Measuring  the  carrier’s  accuracy  as  a  percentage  of  the  total 
number  of  product  movements,  Kodak  awards  15  points  for  100  percent  performance 
and  0  points  for  performance  below  95  percent. 

Customer  Survey 

The  third  major  component  of  Kodak’s  program  is  customer  surveys.  Those 
survey;>  measure  how  relocating  employees  perceive  the  quality  of  the  services  that 
they  received  from  carriers.  The  annex  is  a  copy  of  Kodak’s  customer  survey  form  for 
rating  carrier  performance.  The  point  values  for  the  ratings  shown  on  the  form  are  as 


follows: 

•  ExcellenKvery  satisfied  4 

•  Very  good/satisfied  3 

•  Good/neither  satisfied/dissatisfied  2 

•  Fair/somewhat  dissatisfied  1 

•  Poor/very  dissatisfied  0 
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PROGRAM  RESULTS 


During  the  second  quarter  cf  1992,  Kodak  awarded  520  shipments  to  its 
four  carriers  and  two  agents.  All  520  shipments  were  packed,  loaded,  and  delivered 
on  time,  for  100  percent  scores.  Those  same  shipments  resulted  in  the  claims 
experience  shown  in  Table  B-2.  (Kodak’s  1991  experience  is  also  shown  for  reference.) 

TABLE  B-2 

KODAK  CLAIM  EXPERIENCE 
(2nd  Quarter.  1992) 


Carrier* 

Claims  ratio  (%)b 

Value  of  average  claim  ($) 

2nd  Quarter 
1992 

1991 

2nd  Quarter 
1992 

1991 

1 

2.7 

4.5 

42 

300 

2 

12.5 

14.9 

446 

658 

3 

9.9 

7.4 

316 

355 

4 

18.1 

23.1 

302 

283 

5 

8.7 

13.6 

624 

368 

6 

10.5 

— 

254 

- 

Average 

11.0 

10.7 

361 

412 

*  Includes  four  carriers  and  two  a9ents. 

■>  The  ratio  of  shipments  with  claims  agairm  total  shipmerta,  expressed  as  a  percent. 
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Kodak  Transportation  Quality  Survey 


Dear  Custorner, 


We  at  Kodak  are  dedicated  to  achieving  excellence  in  every  service  provided  to  our  customers. 
Your  input  is  essential  for  helping  us  to  constantly  improve.  You  recently  experienced  our 
equipment  carrier  service.  Will  you  please  take  a  moment  and  fill  out  the  carrier  evaluation 
questionaire  below. 


Thank  you  in  advance  for  sharing 


T.M.  Colomb 


Manager  Domestic  General  Transportation 
Eastman  Kodak  Company 


y^^Jtte)th€9following%yMAetkingUieimppropnateiboxi 


vwr 


fxciuiwT _ COOP _ COOP _ TAit _ eoo» 


Equipn>ent  deiivered/picked 
up  when  expected: 

□ 

□ 

□ 

□ 

Professionalism  of  delivery/ 
pickup  crew: 

□ 

□ 

□ 

□ 

di  (33) 

Efficiency  of  delivery/ 
pickup  crew: 

□ 

□ 

□ 

□ 

Courtesy  of  delivery/ 
pickup  crew: 

□ 

□ 

□ 

□ 

□  a.) 

vimf 

SATlSFlfO 

SATISrilD 

NflTHn 

SATlSrilD/OISSATISrifO 

SOMfWMAT 

DtSSATISrilD 

VWY 

oissATisrico 

Your  level  of  satisfaction  with 
delivery/pickup  service: 

□ 

□ 

□ 

□ 

CH  (71) 

Remarks; 


If  you  are  dissatisfied  with  any  of  the  above,  would  you  like  a  Kodak  representative  to  contaa  you?  Yes  G  No  □ 

(rj) 


f/eaM  nfold  with  tfia  f  wi/Atw  Ktply  Mtll  ptnti  f§elng  out  aaa/  and  mall. 


APPENDIX  C 
MOTOROLA,  INC. 


TITLE 


Total  Cost  Measvrement  System 

PROGRAM  DESCRIPTION 

Motorola  has  an  equivalent  of  a  “best-value”  system  to  evaluate  the  quality  of 
the  products  and  services  provided  by  its  suppliers.  In  operation  since  1989,  the  Total 
Cost  Measurement  System  (TCMS)  is  an  automated  program  for  tabulating  carrier 
performance  data. 

The  TCMS  identifies  the  suppliers  that  provide  the  highest  quality  goods  and 
services,  which  serves  as  an  incentive  for  them  to  provide  even  better  goods  and 
services.  It  also  identifies  the  suppliers  unwilling  or  unable  to  meet  Motorola’s 
quality  standards.  Those  suppliers  are  subsequently  denied  additional  business  from 
Motorola. 

Motorola  currently  uses  three  van  carriers  to  relocate  employees’  household 
goods  and  to  move  sensitive  electronic  components.  A  Transportation  Council, 
including  transportation  managers  from  each  of  Motorola’s  10  major  business  areas, 
negotiates  3-year  contracts  with  the  carriers,  which  may  use  any  local  agent  they 
choose. 

QUALITY  PROGRAM 

The  TCMS  considers  several  performance  factors  and  assigns  weights  (i.e.,  the 
relative  degree  of  importance,  as  judged  by  the  “expert  opinion”  of  Transportation 
Council  members)  to  each.  The  weight  and  the  composition  of  each  factor  are  shown 
below. 

•  Cost  (10  percent) 

►  The  number  of  loss  or  damage  claims  filed,  expressed  as  a  percentage  of 
the  number  of  bills  generated. 
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►  The  number  of  claims  paid  expressed  as  a  percentage  of  the  number  of 
bills  generated. 

•  Quality  ( 10  percent) 

The  percentage  of  bills  that  contained  errors  processed  through  invoice 
payment  companies. 

•  Service  ( 1 5  percent) 

Transportation  Council  members  subjectively  rate  each  carrier  on  a  variety 
of  service  factors,  such  as  proof  of  delivery,  customer  service,  and  carrier 
representation. 

•  Delivery  (65  percent) 

The  percentage  of  on-time  delivery. 

Each  quarter,  Motorola  discusses  the  performance  scores  with  each  carrier. 
Carriers  with  scores  below  1.2  (1.0  is  excellent  and  2.0  is  assigned  to  carriers  that 
provide  unacceptable  performance)  are  eligible  to  compete  for  traffic  and  receive  a 
quality  performance  award. 

PROGRAM  RESULTS 

Motorola  credits  TCMS  with 

•  Reducing  the  number  of  its  suppliers,  including  carriers 

•  Providing  an  incentive  for  good  suppliers  to  improve  their  service 

•  Enhancing  its  internal  quality  program. 
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APPENDIX  D 


XEROX  CORPORATION 


TITLE 

Carrier  Selection  and  Measurement  System 

PROGRAM  DESCRIPTION 

The  Xerox  Corporation  provides  management  and  logistics  services  to  its 
business  elements  through  Via  Xerox ,  a  wholly  owned  subsidiary.  Via  Xerox 
manages  and  oversees  the  relocation  of  approximately  1,000  to  1,500  employees 
annually,  at  a  total  cost  exceeding  $7  million.  It  currently  uses  10  agents 
representing  8  carriers. 

Via  Xerox  prescribes  stringent  quality  standards  for  its  contract  carriers. 
Those  standards  include  full  replacement  of  lost/damaged  items,  guaranteed  pickup 
and  delivery  dates,  and  storage-in-transit.  The  carriers  must  also  designate  a 
dedicated  coordinator  for  Xerox  shipments,  ensure  no  seasonal  rate  increases,  offer 
corporate  discounts  and  frozen  tariffs,  and  demonstrate  geographic  strength. 

The  Carrier  Selection  and  Measurement  System  (CSMS)  -  Via  Xerox’s 
automated  carrier  selection,  evaluation,  and  measurement  program  -  evaluates 
carrier  performance  on  the  basis  of 

•  Customer  satisfaction,  measured  by  the  results  of  surveys  completed  by 
relocating  employees.  (The  annex  provides  a  copy  of  the  survey  form.) 

•  Claims  ratio,  measured  in  tenns  of  the  number  of  claims  against  the  number 
of  moves,  number  of  claims  settled,  and  claims  dollars  paid. 

•  Billing  quality,  measured  in  terms  of  the  number  of  billing  errors. 

Via  Xerox  evaluates  each  of  these  factors  independently;  it  does  not  “roll-up” 
the  results.  That  process  functions  as  follows: 

•  Upon  delivery  of  a  relocating  employee’s  household  goods  to  destination,  the 
employee  completes  the  survey  form;  the  current  return  rate  is  46  percent. 
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•  When  Via  Xerox  receives  a  completed  survey,  it  date-stamps  the  form, 
reviews  the  contents,  and  files  the  results  with  other  simveys  received  that 
month. 

•  At  the  beginning  of  the  following  month,  Via  Xerox  enters  the  results  of 
each  survey  into  the  CSMS  database  and  creates  three  reports: 

►  A  carrier  report  that  shows  carrier  rankings  for  either  the  most  recent 
month  or  year-to-date 

¥  A  Via  Xerox  report  that  is  used  to  compute  the  internal  CSMS  ratings 

►  A  claims  report  that  provides  information  on  the  ratio  of  claims. 

•  Via  Xerox  uses  the  reports  to  compute  an  agent’s  “stack  rank”  (rating  in 
relation  to  the  others)  and  the  percent  of  “ExcellentA/^ery  Good”  ratings. 

The  CSMS  requires  completed  surveys  on  at  least  33  percent  of  an  agent’s 
moves  to  qualify  for  ranking.  Via  Xerox  sends  the  CSMS  results,  accompanied  by  a 
letter  of  issues  or  reinforcement,  to  all  agents  monthly.  It  also  reviews  all  scores  and 
issues  with  the  agents  quarterly. 

Despite  the  three  CSMS  evaluation  areas,  customer  evaluations,  as  documented 
on  the  survey  form,  comprise  the  core  of  the  CSMS  system.  The  company’s  emphasis 
on  customer  satisfaction  stems  from  its  conviction  that  a  satisfied  customer  is  the  key 
to  business  and  profit  growth. 

PROGRAM  RESULTS 

The  CSMS  is  a  new  initiative.  Although  it  has  accumulated  little  historical 
data.  Via  Xerox  has  already  realized  improvements  in  agent  performance  and 
expects  further  improvement.  Through  the  first  8  months  of  1992,  agent  ratings  have 
averaged  between  2.59  and  3.15  (the  target  is  3.30)  and  their  ExcellentA^ery  Good 
ratings  have  averaged  between  59  and  83  (Via  Xerox 's  target  is  85). 
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CUSTOMCR:  _ 

MOVE  ORIGIN: _ 

DESTINATION:  _ 

Survty  Mailtd  to  Customer  on: 

Van  line/Agent: 

CARRIER  COMMITMENTS: 

1 )  Was  packing,  loading,  delivery  and  unpacking  (if  requested)  completed  withm  the  agreed  spread  dates’’ 

□  Yes  □  No 

IF  not,  PLEASE  EXPLAIN: _ _ 

2)  Approximately  what  percent  of  the  boxes,  supplied  and  pacxed  by  the  Carrier,  did  the  Carrier  also  unpack,  if  any? 

*4  Unpacked 

CARRIER  COMMUNICATIONS: 

1)  How  many  work  days  after  you  first  spoke  with  Michele  Cavalien  from  via  Xerox,  were  you  first  contacted  by  the 

Carrier?  _  Work  days 

2)  Please  check  indicating  each  period  below  during  which  the  Carrier  contacted  you 

Pre-Packing/Re-Confirm  bates  Packing  Loading  Delivery  Post  Delivery 

□  □  □  □  □ 

3)  If  you  gave  the  driver,  or  the  Carrier,  your  destination  telephone  number,  did  either  contact  you  the  day  prior  to 

your  aaual  delivery’  □  Yes  □  No 

4)  Please  evaluate  the  overall  quality  of  communications  from  the  Carrier.  (Consider  frequency,  helpfulness, 
responsiveness,  professionalism,  knowledge,  etc.) 

EXCELLENT  \ERY  GOOD  GOOD  FAIR  POOR 

□  □  □  □  □ 

Comments;  _ 

CARRIER  SUPPORT  PERSONNEL: 

(Consider  each's  quality,  efficiency,  professionalism,  responsiveness,  etc.) 


1)  ESTIMATOR 

Comments: 

EXCELLENT 

□ 

VERY  GOOD 
□ 

GOOD 

□ 

FAIR 

□ 

POOR 

□ 

2)  PACKERS 

EXCELLENT 

VERY  GOOD 

GOOD 

FAIR 

POOR 

Comments:  _ 

□ 

□ 

□ 

□ 

□ 

3)  LOADERS 

EXCELLENT 

VERY  GOOD 

GOOD 

FAIR 

POOR 

□ 

□ 

□ 

□ 

□ 

Comments: 

4)  DRIVER 

EXCELLENT 

VERY  GOOD 

GOOD 

FAIR 

POOR 

□ 

□ 

□ 

□ 

□ 

Comments: 

S)  UNLOAOERS 

EXCELLENT 

VERY  GOOD 

GOOD 

FAIR 

POOR 

□ 

□ 

□ 

□ 

□ 

Comments: 

6)  UNPACKERS 

EXCELLENT 

VERY  GOOD 

GOOD 

FAIR 

POOR 

□ 

□ 

n 

□ 

□ 

Comments; 

D.1.3 


FOR  VIA  XEROX  l^SE  ONLY 
Delivery  Date:  _______ 

•/A:  _ 

0/A:  _ 

0/A:  _ 

C.O.O.:  _ 

Driver:  _ 


1)  If  you  *ip«ri«nccd  «ny  loss  or  d«m«9«,  what  would  you  estimate  the  repair  costs  (replacement  cost  if  item  is  not 
repaira^)  to  be? 


2)  Are  you  filing  a  claim  for  this  lossrdamage? 
IF  YES,  have  you  filed  your  claim  yet? 

IF  YES,  has  your  claim  been  settled  yet? 


■■ 

Yes 

■■ 

Yes 

Yes 

No 

No 

No 


3)  Please  evaluate  the  overall  quality  of  the  Carrier's  claims  settlement  process.  (Consider  responsiveness, 
professionalism,  filing  ease,  reasonableness.) 

excellent  very  good  good  fair  poor 

□  □  □  □  □ 


Comments: 


CARRIER’S  OVERALL  PERFORMANCE  EVALUATION:  (Considering  all  of  the  above.) 

excellent  very  GOOD  GOOD  FAIR 

□  □  □  □ 


POOR 

□ 


VIA  XEROX  MOVING  SERVICES’  PERFORMANCE  EVALUATION: 


Coordinator:  Michele  Cavalien 


1 )  Please  check  each  period  below  during  which  you  were  contacted  X*!®* 


initially 

□ 


POST  ESTIMATE/PRE-PACKING 

□ 


POST  DELIVERY 

□ 


2)  Please  evaluate  the  helpfulness  of  any  support  matenai/advice  you  were  given 

EXCELLENT  VERY  GOOD  GOOD 

□  □  □ 


FAIR 

□ 


POOR 

□ 


3)  Did  you  require  Via  Xerox'  assistance  to  resolve  an  issue(s)  with  the  Carrier  (or  one  of^ts  sy|i^rt  personnel)? 


□  Y( 


□  NO 


•  ISSU£(S): 


4) 


Please  evaluate  Via  Xerox'  effectiveness  in  helping  resolve  this  issue(s). 

EXCELLENT  VERY  GOOD  GOOD  FAIR  POOR 

□  □  □  □  □ 

Please  evaluate  the  overall  quality  of  the  support  you  received  from  Via  Xerox.  (Consider  responsiveness, 
professionalism,  knowledge,  follow  up,  etc.) 

EXCELLENT  VERY  GOOD  GOOD  FAIR  POOR 

□  □  □  □  □ 


WE  WOULD  APPREOATE  ANY  ADDITIONAL  COMMENTS  REGARDING  YOUR  MOVE  AND/OR  ANY  SUGGESTIONS  FOR 
FUTURE  “VIA  XEROX*  IMPROVEMENTS  OR  NEW  SERVICE  OFFERINGS. 
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TENNESSEE  VALLEY  AUTHORITY 


TITLE 


None 

PROGRAM  DESCRIPTION 

The  Tennessee  Valley  Authority  (TVA)  relocates  approximately  300  employees 
annually,  with  most  moves  occurring  within  Tennessee  or  to  contiguous  states.  As  a 
result,  TVA  uses  only  a  few  intrastate  carriers,  one  interstate/regional  carrier,  and 
one  national  carrier. 

The  TVA,  which  is  not  subject  to  the  Federal  Acquisition  Regulation,  uses 
5-year  contracts  with  annual  renewal  options.  Its  primary  emphasis  is  to  offer 
shipments  to  minority  carriers,  with  awards  to  women-owned,  small  business,  and 
large  carriers,  following  in  that  order. 

The  TVA’s  carrier  evaluation  program  depends  entirely  on  carrier  performance 
surveys  completed  by  relocating  employees.  It  uses  a  carrier  performance  rating, 
computed  semiannually  from  the  employees’  survey  forms,  as  a  multiplier  of  the 
rates  offered  by  carriers  during  contract  negotiations.  Those  ratings  are  based  upon 
employees'  survey  responses  to  four  performance  factors,  which  TVA  weights 
subjectively  using  expert  opinion.  The  TVA’s  current  weighting  scheme  is  communi¬ 
cations,  20  percent;  service,  40  percent;  appearance,  20  percent,  and  loss/damage, 
20  percent. 

All  carriers  must  show  continuing  improvement  in  all  four  of  performance 

in  order  for  TVA  to  retain  them. 

PROGRAM  RESULTS 

Not  available. 
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APPENDIX  F 


GENERAL  SERVICES  ADMINISTRATION 


TITLE 


Household  Goods  Carrier  Evaluation 

PROGRAM  DESCRIPTION 

The  General  Services  Administration  (GSA)  relocates  approximately 
20,000  Federal  employees  annually  using  266  approved  carriers  over  254  domestic 
routes.  1 

Initiated  in  1989,  GSA’s  program  uses  survey  data  to  compute  a  carrier 
“Performance  Index.”  Based  upon  input  from  relocating  employees,  which  comprise 
70  percent  of  the  index,  and  Government  bill  of  lading  (GBL)  issuing  offices  (the  other 
30  percent),  the  Performance  Index  in  turn  feeds  into  a  carrier’s  overall  "Value 
Index,”  which  GSA  computes  for  each  carrier  over  each  route  it  serves.  The  Value 
Index  consists  of  the  Performance  Index  (70  percent)  and  the  carrier’s  offered 
"Discount  Index”  (30  percent),  which  GSA  computes  by  dividing  the  carrier’s  bid  rate 
by  the  average  carrier  bid  rate. 

The  GSA  program  has  several  unique  features: 

•  Its  evaluations  cannot  be  argued  or  appealed  by  carriers  because  the 
responses  are  subjective. 

•  It  requests  information  on  some  factors  that  many  carrier  evaluation 
programs  do  not  consider,  such  as  workers’  courtesy  and  responsiveness  to 
problems.  (See  the  annex  for  a  copy  of  the  form  that  GSA  uses  to  collect 
carrier  performance  information.) 

•  It  uses  expert  opinion  to  determine  the  weights  of  the  various  evaluation 
factors,  subject  to  negotiation  with  carrier  representatives. 

.  •  It  does  not  provide  for  any  formal  or  equitable  tonnage  distribution  to 
carriers. 


1  According  to  GSA  procedures,  each  of  the  266  carriers  must  be  a  principal  operating  company. 


•  It  operates  on  an  annual  evaluation  cycle. 

•  It  sets  high  minimum  Performance  Index  standards  -  95  percent  in  1992, 
97  percent  in  1993,  and  100  percent  in  1994. 

In  addition,  GSA  does  not  compute  a  Performance  Index  for  carriers  that 
serviced  only  one  shipment  or  generated  only  one  survey  form  during  the  evaluation 
period;  it  requires  a  minimum  of  two  in  each  case. 

PROGRAM  RESULTS 


Not  available. 
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HOUSEHOlD  GOODS  CARRIER  EVALUATION 


PRIVACY  ACT  NOTICE;  Tht  Information  on  thli  form  to  soljdted  under  Title  38,  United  States  Code,  and  will 
be  used  to  monitor  and  control  the  household  goods  carrier's  performance.  The  Information  may  be  furnished 
the  cirrler  Involved  for  hli  evalustloa  Your  disclosure  of  tlds  Information  is  voluntary  and  will  eld  In  our  overall 
mieeion  of  making  certain  transfereoa  receive  satisfactory  performance  In  the  shipment  of  their  household 

goods.  _ _  •  ■ _ 

INSTRUCTIONS:  Complete  this  forrh  upon  dslivery  of  household  goods  to  new  duty  stations.  Then  send  to 
your  GBL  Issuing  Officer  for  their  evaluation  and.  submittal  to  GSA,  ATTENTION  GBL  ISSUING  OFFICER:  Once 
you  have  completed  your  evaluation  of  the  carrier,  men  to  Genenl  Servlcee  Admlnlstration-5Fa5^  1500  East 
Bannister  Road,  Kansas  City,  MO  B4131.  _ ^ 

N«M(  AND  raiS^NT  AOOaESS  OF  fiMftOTEE  ~ 


CUallENT  auSiNESS  TELCPHONt  NO. 
CASRiaa  6n  cbl 


H6Klf  tfLtrKONE  NUMttK 


CSL 


lOu)  duty  station 


NEW  DUTY  ST  ATION  uneiuot  ZIP  Coa*> 


riCKur  DATE 


OSA  CONTROt.  NUMBER 


delivery  date 


EBTTMATE  Or  ANY  lobs  or  DamaI 


RELOCATIN3  EMPLOYEE'S  RESPONSE;  How  would  you  rat*  your  Mitltfsetlon  wKh  th*  tmrtmrl  (Circt*  r*»pc>n»B> 


V*ry 


Somawhw 


IsiaRhar 

Unsiicfiaa 

NDr 


UnsaTUfMd  Unsitlsrieil  Sal  lifted 


SorrtBwhai 

Satisfied 


Very 
Sbi  is  fled 


Qualtiy  of  pDOKine 

OetWttiir«9  hems  wnn  Mil*  or  no  dsrrwge 
Havirtg  workers  vrho  snow  persorisi  eounesy 
Oetwermo  wUiisr  the  scheduled  tme  trime 
Sleerly  communceting  the  services  ie  oe  provided 
Bewg  responsive  in  reaeHtrTp  proOtemj 
How  woutd  you  rete  the  overall  queiity  of  service 


1 

1 

1 

1 

1 

1 


3 

3 

3 

3 

2 

3 

3 


3 

3 

3 

3 

3 

3. 

3 


4 

4 

4 

4 

4 

4 

4 


5 

5 

5 

5 

6 
5 
B 


QBL  ISSUWO  OPPICBR’S  RESPONSa  Mow  would  you  rate  your  evtlsfactlen  wHh  the  eeirlert  (Cirple  response) 


Hevwg  courieous  persons  help  you  when  trecnp  •  shipment  t 

KeepRip  you  Infonried  of  eny  chenyes  occurring  durinp  the  move  1 

Behtg  flexible  et  meetng  speciei  employee  or  egency  needs  1 

How  would  you  rate  the  overslt  queliiy  of  the  service  t 


2 

2 

2 

2 


3 

3 

3 

3 


4 

4 

4 

4 


5 

5 
B 

6 


COMMENTS:  Uee  •dditleiihl  eheets.  If  neoeeeery. 


tTgBUlWi  W  TO^ixyvn - -  - - 

Bin - 

STONATORE  or  OBL  IttUiNO  OfridilK 

cin - 

TSllfHONE  NUMSSR 

OENERAL  services  AOMNISTRATON  OBAPOhMSoaoiREV.  e-iM 


XNVia  3ovd  ONiaaoayd 
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